Action of overalgebra in Plancherel decomposition and 
shift operators in imaginary direction 

Yurii A. Neretin Q 

In last 50 years, there were solved many problems of explicit spectral de- 
compositions for restrictions of unitary representations to subgroups (see the 
bibliography in p2f ). 

This class of problems also contains the following problems, which are for- 
mulated in other terms. 

1 . Decomposition of the tensor product of representations p\ , pi of a group 
G. Indeed, this is exactly the problem of restriction of representations of G x G 
to the diagonal subgroup G C G x G. 

2. Decomposition of L 2 on a pseudo-Riemannian symmetric space G/H. As 
was observed^ in |p3| , for each classical pseudo-Riemannian symmetric space 
G/H, there exists a canonical classical group G D G and a representation p 
of the group G of degenerated principal series satisfying one of two following 
properties (usually the first variant is realized): 

— the restriction of p to G is L 2 (G/H) 

— the restriction of p to G is the direct sum of the spaces L 2 (G/Hj), there 
G/Hj is a finite collection of symmetric spaces, and G/H is one of the spaces 
G/Hj. 

Hence the decomposition of L 2 on a classical Riemannian symmetric space 
can be considered as a restriction problem. 

Description of the spectral type (without explicit Plancherel formula) for all 
the pseudo-Riemannian symmetric spaces was recently obtained in the works 
of van den Ban, Schlichtkrull, Delorme, and Oshima (the proof is contained in 
the union of a large collection of papers, for references, see ||, 0). It seems 
that for classical symmetric spaces the problem of evaluation of the Plancherel 
measure is near the final solution. 

For some cases, the explicit Plancherel measure is known; in particular, for 
L 2 on semisimple groups on Riemannian symmetric spaces (|J, Jl0| , 

[ p"l| , sec also [^5)), on rank 1 spaces (pC||, and on the spaces Gc/Gr, where 
Gc is a complex group, and Gr is its real form (|Hq], PI). 

3. Berezin kernel representations (deformations of L 2 on Riemannian non- 
compact symmetric space G/K) also can be obtained by the restriction from 
some overgroup G° D G, see p4| , p6| ] and references in p6[ . 

Usually the problem of the noncommutative harmonic analysis is formulated 
as the problem about the spectrum of a representation or as the more compli- 
cated problem of an explicit decomposition of a representation into a direct 
integral of irreducible representations; the last question includes the explicit 
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2 This phenomenon could be easily observed from the Makarevich paper jis) published in 
1973, in a strange way it remained nonformulated for a long time 
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evaluation of the spectral measure (the so-called Plancherel measure). Recent 
works of the author p5[ , |2l| contain an attempt of an investigation of "the 
analysis after the Plancherel formula" . 

This work is a continuation of pBJ. Here we are trying to understand the 
answer to following question. 

Question. Assume that we know the explicit Plancherel formula for the 
restriction of a unitary representation p of a group G to a subgroup H . Is 
it possible to write the action of Lie algebra of G in the direct integral of 
representations of HI 

We obtain the positive solution of this problem for one of the simplest pos- 
sible examples, precisely, for the tensor product of a representation of SL(2,R) 
with a highest weight and the conjugate representation with a lowest weight. 
This tensor product and its decomposition were widely discussed in the lit- 
erature on representation theory and special functions in last 40 years (some 
references: [|27), [§, ||], j§). Nevertheless, the formula for the action 

of sl% ® SI2 in tensor product were not appear. The reason is the unusual for 
representation theory form of these Lie algebra operators. 

It turns out that the operators (12)-(14) of Lie algebra sh © SI2 are the 
second order differential operators with respect to one variable and the second 
order difference operators with respect to the another variable; moreover, it 
turns out that the difference operators are defined in the terms of a shift in the 
imaginary direction, i.e., in our formula, there appear the operators of the form 



(i 2 = —I). There is no self-contradiction in this expression, the shift operators 
are well defined on functions admitting the holomorphic continuation to an 
appropriate strip. Operators / 1— ► xf and / 1— * -4- also are not defined on the 
whole L 2 (R). 

First, these Lie algebra operators were obtained by author using theorems 
on the operational calculus for the index hypergeometric transform (17) from 
[ p5| . But the final formulas are elementary and admit a direct verification by 
elementary tools. 

Since the approximate structure of formulas for action of overalgebra now 
became more clear, it is natural to formulate the general problem given above. 

This work was prepaired during my visit to the Erwin Schrodinger Institute 
(Vienna). I thank the administrators of the Institute for their hospitality. 

1. Group SL(2,R). We realize the group SL(2,R) as the group of complex 
2x2 matrices having the form 



By D we denote the disk \z\ < 1 on the complex plane C, by S 1 we denote 
the circle \z\ = 1; we represent the points of the circle in the form z = e %<p . 



Tf(x)=f(x + i) 



for / G L 2 (R) 
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The group SL(2,R) acts on the disk by the Mobius transformations 

a b\ az + b 

- _ : z i ► = . (1) 

b a) bz + a V ; 

If z G S , then its image under (1) also is contained in S . 

2. Highest weight representations of SL(2,R). Fix a > 1. Consider 
the space H a of holomorphic functions / on the circle D satisfying the condition 

/ |/(z)| 2 (l-|z| 2 r- 2 {dz}<oo, (2) 

where {dz} denotes the Lebesgue measure on D. Define the inner product in 
the space H a by 

</, g) a = — [ f(z)W)a - M 2 )^ 2 {d4; (3) 

the pre- integral factor provides the identity (1, 1) Q = 1. In terms of the Taylor 
coefficients, the inner product is given by 

(£ c k z\ 2 c' k z% = E c k c' k a(a + 1) k \ a + k _ iy 

and condition (2) for f(z) = Yl c kZ k has the form 

^\c k \ 2 k- {a -V < oo. (4) 

It can easily be checked that the space H a is complete with respect to the 
norm defined by this inner product, i.e., H a is a Hilbert space. 

The group SL(2,R) acts in the space H a by the unitary operators 



First, consider the case, when a is an integer. The factor (bz + a)~ Q is a 
degree of the derivative of the function (1), this implies that T a is a represen- 
tation: 

T a (gi)T a (g 2 ) = T a (g 1 g 2 ). (6) 
The operators T a (g) are unitary, i.e., 

(T a (g)f 1 ,T a (g)f 2 ) a = (f 1 ,f 2 ) a , 
this can be easily checked by a change of the variable. 
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For a noninteger a, T a {g) also is an unitary representation; we only must 
explain the meaning of the expression 

(bz + a)- a = (l + 6a- 1 z)- Q a- a = (1 + ba' 1 z)- a e- a ^ na+2nik \ (7) 

Obviously, 1 6cx ~ 1 1 < 1. Hence the function 

(1 + ba-'zT a := 1 + ^ba-'z + ^^(ba^fz 2 + ... 
is well defined. 

Therefore the operator (5) is defined up to the factor e _2,rli;Q , the absolute 
value of this factor is 1. Hence equality (6) is replaced by 

(gi)T a (g 2 ) = e-T a ( gi g 2 ), 

where |0| = 1. Thus, T a (g) is an unitary projective representation of the group 
SL(2,R). 

Remark. Clearly, T a (g) can be also considered as a linear representation 
of the universal covering group of SL(2,R). 

3. Tensor product. By T a (g) we denote the representation complex 
conjugate to T a (g), it acts in the space H a of antiholomorphic functions in the 
circle \u\ < 1 by the formula 

t 4 J)'«-'(!Ei)<" + ->~- 

The inner product in H a is given by (3). 

Consider the space H a <g) H a . It consists of functions f(z,u) on the bidisk 
DxD, holomorphic with respect to the variable z and antiholomorphic in u; 
the inner product in H a ® H a is given by 

</l,/2> = ^-5^ II h{z,u)Uz^{l-ZZ) a -\l-UUr-\dz}{du}. 
I" J J DxD 

The group SL 2 (K) x SL 2 (M) acts in H a ®~H a by the operators T a ( gi ) ® T a (g 2 ) 
given by 

\bi aij \b 2 a 2 J 

fcuz + h a 2 u + b 2 \- 

= /(= —'7^^ ){biz + ai) a (b 2 u + a 2 ) . 

\biz + a 1 b 2 u + a 2 J 

We restrict this representation to the diagonal subgroup SL 2 (IR) C SL 2 (M) x 
SL 2 (R), this corresponds to the substitution ai = a 2 = a, b x = b 2 = b to the 
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last formula. This representation of SL2(M) is a linear (nonprojective) repre- 
sentation. Indeed, 

(bz + a)- a (bu + a)- a = {l + a- 1 bz)- a (l + a- 1 bu)~ a {aa)- a , 

and this expression is a single- valued function for \z\ < 1, |it| < 1. 

4. Principal series of representations. Fix set. Consider the repre- 
sentation p s of the group SL(2,R) in L 2 on the circle S 1 given by 



6e 1 ^ + a 



These unitary representations are the so-called representations of the principal 
series. Recall, that the representation T s is equivalent to T_ s (see ||). 

5. Spectral decomposition. Let a be the same as in pp. 2-3. Consider 
the kernel 

(l-ze i * , )- 1 / 2 - is (l- ue - i * , )- 1 /2-" 

K a ((p, S\ Z.U) := ; 777; — ; , 

where |z| < 1, \u\ < 1, <p G [0,27r], s ft. Consider the integral operator Jq,, 
that takes a function / £ H a ® i7 Q to the function F(y>, s) given by 

F(<p,s) = JJ K a (ip,s;z,u)f{z,u)(l - zz) a - 2 (l ~ uu) a - 2 {dz}{du}. (8) 

DxD 

Consider the action of the group SL(2,M) in space of functions in the vari- 
ables (ip, s) defined by 

,„„w i 

il P J- 77l _1_2 * s 



R (l b )F(e^,s) = F(^^,s)\be^ + a\ 



Note, that for a fixed set the function F(e l(p , s) (as a function in ip) transforms 
by the formula for principal series representations. 

Simple calculation show, that the operator J a intertwines the representations 
T a {g)®T a {g) and R of SL(2,R): 

J a -(T a {g)®T a (g))=R(g)-J a (9) 

Theorem 1 . Operator J a is a unitary operator from H a eg) H a to the space 
L 2 on ip G [0, 2tt], s with respect to the measure 



T(a- 1/2 + is)\ 2 sh(7rs) T(a - 1/2 + is)T(l/2 + is) 



s —— — - ds dip 



T(a) 2 ch(vrs) T T(a)T{is) 
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ds dip. 
(10) 
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Thus T a eg) T a is a multiplicity free integral over principal series represen- 
tations (this fact was obtained by Pukanszk y fl27| ). Various ways for obtaining 
the Plancherel measure (9) are contained in p9|, Q , || , pj| . 

6. Holomorphic continuation of J a f(cp,s). Denote by W the space 
of all / 6 H a (g) H a admitting analytic continuation to a bidisk \z\ < 1 + 5, 
\u\ < 1 + 6. 

For / G W, consider its Taylor series 

f(z,u) = ^2c k iz k u l . 

k,l 

Obviously, its coefficients Cki exponentially decrease for k + I — > oo. 

Lemma. Fix (p. For f G W , the function J a f(ip, s) can be extended holo- 
morphically to the whole complex plane s G C. 

Proof. Fix s G C, <p G [0, 2ir}. The function K a (ip, s; z, u) as a function 
in the variables z, u has a polynomial growth near the boundary of the bidisk 
D x D: 

\K a (<p, s; z, u)| < exp{4vr (1 + |Ims|}(l - |z|)-< 1+Q+2 l Rcs D(l - | u |)-(l+«+a|H«l) 

Hence, for fixed s, (/?, the coefficients a k i — aki(<p, s) of the series 

K a (ip,s;z,u) = y^^a k i(s,ip)z k u l 

have the polynomial growth as k, I — > +oo. 

Indeed, let q(z,u) be a function in the bidisk satisfying 

^C-tT' 1 for |«| < 1 - S, \u\ < 1 - S. 

The Taylor coefficients b^i of q(z 7 u) are given by the formula 

q(z) dz du 



|2|=l-5,|u|=l-<5 

Hence, 

|6 fcI | < const -5- h il-5)- k - 1 - 2 . 

for all 5. We choose S = h/(h + k + l + 2) and obtain polynomial growth for bki ■ 
For the kernel K a (z, u), we obtain in this way the uniform estimates of the 
form |djy| ^ A ■ (1 + k + l) T in each rectangle 

|Ims|<JV, |Res|s$M. (11) 

Since the Taylor coefficients Cki for f £ W exponentially decrease, the series 

J a f(cp, s) = c k ,ia k ,i(s, (p)- 
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is absolutely convergent (see. (4)), the summands arc holomorphic with respect 
to s, and the series is uniformly convergent on rectangles (10). 

7. Correspondence of differential operators. The operators of the Lie 
algebra sl(2) © sl(2) in the space H a ® H a have the form 

d r i z \ d 



rW + nx . T {z) 



L<»> 



r(«) -2^, - 

L_{ = u — + ait 
ou 



For us it will be more convenient the following collection of the operators 



Ln :=L 



M :=4 2) +4 U) ; 



r r(«) 

M_! :=4 u) +lL1; 



t r( z ) r(«). 

_Li .— L x — L_ x , 

M 1 := L[ z) + if"]; 



The operators Lq,Li, L_i span the diagonal subalgebra s?2 in s?2 © s^2- Their 
images under the operator J Q are defined by the formulas 



J a o 


- d _d~ 


f(<P, s ) = 








J a o 


r 2 ^ 

w — h aw 

L cm 


d 

dz 


f(<P, s ) = - 


zcV 2 


•4 f(<P,s) 


J Q o 


r 2 ^ 

-2 7T- + az - 
- oz 


d - 
du- 






-(I+ ia ) e -*" 


/(</>>«) 



These three formulas easily follow from (8). 

Theorem 2. The unitary operator J a transform the operator M to the 
operator 



Qof(<P,s) 



(-±+i«)(-a+§+t«) 



2is 



f(ip,s + i) + 



+ 



(| + is)(a - h + is) 



f{<P,s-i) - 7T 



a + i + is d 2 



2is "' 2is{-\ + is) dtp- 

i.e., for any f G W (see our Section 6 ), 

QoJaf = J a M f. 

The operator Mi transforms to 



■f(<p,8 + i), (12) 



Qif(f,s) = 

(i +is)(-a+ i +is) 



= e 



2is 
l 



f^s +i ) + ^ +is ^r h+is) 

Its 



T f{p,s + i) + j-^- — ffas + %) 



2is(-\ +is) dp 2 



\ + is idip J 



f(<p,s-i)- 
, (13) 
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and the operator M_i transforms to 



Q-if(<fi, s ) 



(| + is)(-a + \ + is) (i + is)(a - \ + is) 
J((fi, s + ^) + ^- 



2is 

-a+\ + is d 2 



2is 



-a+z+is d 



4+ is idtp* 



f(<p,s-i)- 
■ (14) 



PROOF. These formulas can be checked by direct calculations. For instance, 
let us consider M . 

Obviously, the operator M is selfadjoint in H a ® H a . Hence 

J a M f(ip,s) = 

K{ip, s; z, u) [(z + u-^ +a)] /(*, 5) • (1 - zz)«- 2 (l -ml)"- 2 {du} = 



DxD 



//[( 
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z— +u- — ha 



K(<p, s-z, u)-f(z,u)(l-zz) a - 2 (l-uu) a - 2 {dz} {du} 



DxD 



Thus the first statement of the theorem is equivalent to the identity 
d 



/ U \ 

\du + ^dz 1 + / S ' ^' ^ ~ ^° ^ ^ s; = 
After division by K, this identity transforms to the form 



(15) 



K- 1 {u^- + z^-\K + a + K 
I ou oz> 

_ 1 {l+is)(a- §+»«) 



(— | + is) (—a + rj + is) 



2is 



#(« + *) - 



K -iv2 a_ — 2 J -K(s-i)+K- 1 — 2 ' ;~ " K(a + i) (16) 



_ x -a + | + is d 2 



2is 



2is(-\+is) d^p 2 ' 



The function in the left side is a long rational expression in z, u, e lip , s; the 
identity can be easily verified by MAPLE. 

Let us explain how to verify the identity (15) "by hands". Each summand 
of (15) can be represented as a linear combination 



a(s) + b(s)- 



1 



1 — zu 



■c(s). 



1 



1 - ze l f 1 - ue~ % f 



-As)- 



(1 - ze^){l -ue-^) 
1 — zu 



After this, it remains to sum the coefiitients. 

Remark. Let us apply the operator Q to the functions f(<p,s) = g(s). 
Then the equations 

Q g(s) = (k + a)g(s) 
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coincide with a partial case of the difference equations for the continuous dual 
Hahn polynomials (see |Q, (6.10.9)). 

8. Some remarks. In the work of the author there were obtained 
some elements of an operational calculus for the index hypergeometric transform 
(it is called also by Olevsky transform or Jacobi transform, see pofl , [ EjJ ) 

1 f°° 

g(x) i ► g(s) = / g{x) 2 F 1 (b + is,b- is; b + c; -x)x b+c - 1 (l + x) b - c dx, 

r (o + c ) Jo 

(17) 

In p5[ , it was shown that the index hypergeometric transform maps the differ- 
ential operators 

d 

Ag(x) = xg(x); Bg(x) = z— 

(and hence all the operators admitting polynomial expression in x, x-B^) to 
difference operators in imaginary direction; see also the work of Cherednik || 
containing some similar statements for simmetric functions in multidimensional 
case. Existence of the formulas (11)— (13) more or less follows from these results, 
but this way for obtaining the expressions (11)-(13) also is not very simple. 
There arises the following question. 

Question. Is it possible to write explicitly operators of the overalgebra 
for the case of L 2 on a pseudo-Riemannian symmetric space and for the kernel 
representations? 

Is it possible to do this at least for rank 1 symmetric spaces? 
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